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ABSTRACT 


In the design process of interactive sound walk art 
installation, the composer must think his sound design in 
a real or a virtual space according to the listener’s 
promenade. For a given piece, the sound designer leaves 
to the interactor a certain amount of freedom and to the 
system, according to its level of self-sufficiency, a certain 
amount of autonomy. This paper presents an analysis of 
this fundamental composition choice between 
determinism, emergence and interactivity. In the first 
part, this problem is stated as the position of an art piece 
in the D.I.S. (Designer/Interactor/System) triangle. We 
analyze from this point of view several interactive pieces. 
The second part concerns mainly the design process of 
The Listening Walker scripted versions. This interactive 
sound walk relies on the technology and the design 
principle of video games. In the scripting style, the 
designer takes the point of view of the interactor, who 
becomes the narrator. We conclude by an introduction to 
the second version of the same piece written according to 
the emergent style. In this case the designer takes the 
point of view of Non Player Characters controlled by the 
system. 
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1. INTRODUCTION 


One of the authors of this paper has worked as a 
composer on numerous sound walk art installations. 
During the last decade, her work has been influenced by 
the design methods used in video games, leading to 
interactive pieces. In the design process of such pieces, 
as compared with interactive music works, the composer 
think his spatial sound design in a real or a virtual space 
according to the listener's promenade. Such an 
installation shows the evolution of the composer’s role 
from a deterministic creation to a non-deterministic one. 
For a given piece, the composer leaves to the interactor a 
certain amount of freedom and to the system, according 
to its level of self-sufficiency, a certain amount of 
autonomy. According to these two parameters each art 
piece can be positioned somewhere within the triangle 
DIS. (Figure 1) which vertices are the Designer, the 
Interactor and the System. 
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Our main purpose is to analyze the goals and the needs 
of the designer of such interactive pieces, in terms of 
methodology and tools. We first analyzed a set of 
interactive pieces designed by musicians, visual artists 
and game designers. From this analysis, we determined a 
set of classification criteria in order to position these art 
works in the D.LS. triangle. 

A position in this triangle can be related to a choice 
between two writing styles, the “scripting style” and the 
“emergent style”. In the scripting style, the designer takes 
the point of view of the interactor, who becomes the 
narrator. In the emergent style, the designer takes the 
point of view of Non Player Characters (N.P.C.). 

For a deeper analysis, we have designed two versions of 
the same interactive sound installation The Listening 
Walker. 

This paper is mainly devoted to the analysis of the 
design process of the scripted version according to a 
choosen position in the triangle. The design of the 
emergent version is briefly introduced in the conclusion, 
a full description of our work can be found in [1]. 


2. D.LS. TRIANGLE 


2.1 Components 


We have placed the protagonists concerned - namely the 
designer, the interactor and the computer system (1.e. the 
agents in the virtual world) - equidistantly in relation to 
one another. As a first step we outline the following 
aspects relating to each of these three protagonists: their 
knowledge of the environment, their patterns of 
perception, their controls and intervention rules, and their 
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Figure 1. D.LS. triangle 


motivations. 


2.1.1 The designer 

We use here the word designer for an author such as a 
composer, a sound designer, or more generally a creator 
of a sound installation. 

The designer in the triangle has a leadership position. He 
must choose the position of his work in the D.LS triangle. 
Our work relies on the following hypothesis on the 
designer goals: 

e His goal is to abandon linear writing towards a 
better distribution of the "narrative" between the 
designer, the interactor and the system. 

e He wants also to maintain artistic consistency 
within such an open compositional proposal. 

¢ From the designer point of view, the interactor is 
unpredictable. The Designer wants to let a part 
of the story to rely on his objectives, his 
decision-making, his personal perception and 
understanding of the sound space and, more 
generally, the surrounding world. 


2.1.2 The interactor 

Depending on the device and the interface offered, the 
interactor receives visual, auditory, tactile and, more 
rarely, olfactory information. We focus on listening and 
the understanding of space. We assume that the interactor 
discovers the environment in which he moves based 
principally on his auditory attention; control of the 
system and its rules are learnt through trial and error. 

The interactor has many ways to take a decision. 
Decisions may be based on logic or they can be actions in 
response to the interactor’s understanding of a situation. 
We start from the assumption that this is no set goal; 
rather the motivation is toward discovery of an 
“unknown”, in every sense of the word. 


2.1.3 The system 

The interactor is not the only source of indeterminism in 
the execution of the piece. The computer system - by 
combinatorial developments, the use of hazards or any 
other generative technique - can make the execution of 
the sound piece to some degree unpredictable. Usually in 
generative artistic creation, such as in video game, the 
goal of artificial intelligence - and more broadly 
generative mechanisms - is to generate credible 
perceptive behavior. The Designer who is, in general, 
neither a specialist in artificial intelligence, nor a trained 
psychologist defines this role. It therefore seems 
reasonable to consider that the description of these 
components is mainly behavioral. 
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2.2 Interactive art works analysis 


2.2.1 Analysis criteria 

We have selected a set of interactive art pieces in order to 
understand the position choosen by their designers within 
our D.LS. triangle. To do this we have established a set of 
analysis criteria: 

e Interactive or not: either the spectator is an 
interactor, his actions have a real impact on the 
piece execution or not. 

° Endogenous or exogenous: in endogenous pieces 
the indeterminism is internal to the computer 
program while exogenous pieces behavior is, at 
least partly controlled by the interactor.. 

e Exploratory or ontological: in exploratory mode, 
the only ability of the interactor is to control his 
perception of the virtual world (camera and 
listening point) without influencing the 
simulation whereas in ontological mode, he 
interferes with the very existence of this world 
and can change its evolution. 

e Emergent: evolution of a system from the 
interaction between agents in ways that are not 
completely controlled nor fully provided by the 
designer. 

e Awareness or engagment: a conscious 
interactivity does not need the interactor to 
control rendering while engagment requires 
interactor control in learning the rules of 
interaction as in the case of video game. [14]. 


2.2.2 Selection criteria 
The common point between all the presented pieces is the 
will of the designer to create an open form that can be 
perceived in many ways. They are all digital works. 
Some are based around navigation proposed to the public 
both with or without the opportunity to interact with the 
device. Some, due to autonomy granted by the designer 
to the computer system, are generative pieces. Others 
allow communication between the system and the public. 
The privileged form is the installation, in the sense of a 
specific device proposed to the public. Technical and 
spatial complexity of these devices are variable, ranging 
from a computer screen with a keyboard to a system 
deploying in space several screens and a complex system 
of sound diffusion, or else to a soundwalk with 
headphones and sensors. A work in this form can be 
experienced either online or in real space. Some works 
can be considered as interactive writing systems or video 
games, others as sound installations taking after the world 
of video games. Some of them are sound pieces other are 
"silent" pieces. 


Interactor 


Figure 2. Positioning pieces in the triangle D.IS. 
(1.Portrait N°1, L.Courchesne; 2.Le passeur, C. Le 
Prado; 3. Vocatifs, C.Le Prado; 4.Le Jardin des hazards 
“Mutations “, B. Gortais, G. Hutzler,Y. Orlarey; 5.Le 
Jardin des hasards “Divercity“, B.Gortais,G.Hutzler, 
T. Delor, M. Broilliar; 6.Khorwa, M. Malt; 7.Listen 
Lisboa, C. Le Prado; 8.Façade, M.Mateas, A.Stern, 
J.Grieve; 9. Bandonéon, X. Boissarie, R.Cahen, 
A.Schmitt) 


From available informations in exhibition catalogs, 
scientific papers and other data from the web, in certain 
case our personal experience of the pieces, we deduced a 
value for each analysis criteria. We positioned each Art 
works within our D.LS. triangle. This translates to a 
position on each of two axes (designer / interactor and 
designer / system) and, by deduction, a possible 
communication between interactor and system. 

The art works, within which the designer gave both the 
interactor and the system a high degree of autonomy, are 
at the bottom of the triangle and equidistant between 
interactor and system. 

Works with exogenous interactivity may solicit 
different types of attitude and consciousness from the 
interactor. For example, Le passeur [2] and Mutations 
[3], both require a limited awareness of the actions of the 
interactor. The first piece does not solicit the system 
during its exhibition, while the second piece focuses on 
it. Listen Lisboa [4] and Bandonéon [5] share the fact 
that they both invite visitors to engage in interactive 
exploration, being aware of their actions on the evolution 
of audio and visuals, but with no real need to control it. 
Portrait # 1 [6] or Vocatives[7] lie on the boundary 
between exploratory or ontological invitations for the 
interactor. 

Khorwa [8], throught the autonomy and internal 
interaction between agents of the system, has points in 
common with Facade[9], Mutations and Divercity [10]. 
However Khorwa is the only piece that does not require 
any interaction from the listener. Façade is a narrative 
piece; Divercity is more formal. In Façade, the designers 
began by giving a great autonomy to the interactor and 
agents. Then to control the drama while retaining 
autonomy, they added a Drama Manager. For Divercity, 
the process was the reverse. Designers, after several art 
works in which the role of the intercator was almost 
nonexistent alongside a very autonomous system (“Quel 
temps fait-il au Caplan”, “Le jardin des hasards”), added 
the ability for the interactor to communicate with 


autonomous agents. An interactive mat allows the 
interactor to influence the behaviour of the system and 
the audio-visual évolution. 

Through this analysis we observe the difficulty in 
defining a unique position for each piece while at the 
same time the value of this ambiguity. Indeed, between 
the intentions of the designer on an open, scalable 
implementation, the possible autonomy given to the 
system or the interactor, and the shape of the resulting 
work (the type of interactivity on offer, the awareness or 
commitment sought from the interactor), there are many 
possible branches half-way between control and 
autonomy. In the next paragraph, we express this problem 
in terms of writing style. 


2.3 Two writing styles 


" A form is aesthetically valid precisely to the extend that 
it can be seen and understood from multiple perspectives, 
when it exhibits a wide variety of aspects and resonances 
without ever ceasing to be itself." Umberto Eco [11] 


It seems to us important to comprehend the implications 
upon design and realization that result from choosing a 
position within this triangle. To analyze this, we designed 
and developed two versions of the same piece (The 
listening walker). These two versions are distinguished 
by their writing style. One is written in a style we are 
calling “scripted”: an interactive narrative structure in a 
controlled in space and time. The other is written in the 
style we are calling “emergent”: in this case, the meaning 
of the play and its narration emerge from multiple 
possible interactions between the interactor and the 
system. 
This work relies on the two following concepts: 

e To let this unpredictability meaningful for both 
the interactor and the designer, we assume that 
the emergent property is beared by characters, 
which behaviors are defined by the designer and 
can be understood by the interactor. According 
to the game terminology, we call thereafter Non 
Player Characters (N.P.C.). According to the AI 
terminology, they also can be considered as 
agent. Their decisions and actions abilities can 
be purely reactive but also depend on a memory 
of previous steps [W002]. The spatial extent of 
knowledge, perception and memory of their 
environment is not fixed a priori. 

e Then, in order to design the two versions of the 
“Listening Walker”, we stated, as a working 
hypothesis, that the designer can take the place 
of one of the two other protagonists in the DIS 
triangle. He may take the point of view of the 
interactor to build all the possible itinaries and 
stories that can be experienced by the interactor. 
In the opposite, he can also write his piece from 
the perspective of N.P.C., and describe a virtual 
world in which the interactor will evolve. 


2.3.1 The Scripting Style 

An interactive narration is written according to the 
scripting style if the designer takes the interactor’s point 
of view. He is the narrator of an interactive scenario, 
written using a procedural form. The rules are contextual 
and related to a relative chronological order determined 
by the path the interactor follows. In other words, any 
variability inside the narration is linked to the interactor’s 
previous experience, and in particular, his trajectory. 
Each N.P.C’s behavior is described in relation to this 
experience at a given point and time of the interactor’s 
path. The scripted design writes the story from the 
interactor’s point of view. It is the method used for most 
action adventure games. The universe and the events 
exist only when they have been or are perceived by the 
interactor. The interactor is the narrator and he holds in 
his memory the events involved in his narration. 


2.3.2 The Emergent Style 

An interactive narration is written according to the 
emergent style if the designer takes the point of view of 
the system through each N.P.C. An N.P.C. is a potential 
narrator. He has his own perception and memory, which 
influence his behavior, described in a generic (non 
contextual) way. 

In the emergent approach, each N.P.C. has its own 
memories as a result of their perception system. The story 
is described with respect to what has been perceived and 
remembered by each actor (interactor and N.P.C.). 

In both cases, it is possible, from a narrative point of 
view, to simulate a universal narrator who has the 
knowledge of events that no character has seen or heard. 


3. THE LISTENING WALKER SCRIPTED 
STYLE VERSION 


The Listening Walker is an art piece produced as part of 
the Terra Dynamica project, funded by the French 
Government. The purpose of this three years project, 
begun in 2010 and developed by a dozen of academic and 
industrial partners, is to bring life to Terra Numerica, a 
static virtual city. This virtual city (Paris in the 
experiment) is used for numerous applications from 
safety simulation to art installation [12]. 

The production of an Art Piece, the scripted version of 
The Listening Walker, using the Terra Dynamica 
technology, was one of the tasks assigned to the CNAM. 
The title comes from a book by Michel Chion [13]. The 
installation was created in Paris during the “Futur en 
Seine” festival in June 2013. 

The goal of this sound walk is to discover a virtual 
district of Paris around the Pantheon. The interactor 
walks around this part of the city wearing headphones. 
The experimentation takes about ten minutes. This 
installation is designed as a video game with different 
levels of exploration. The interactor’s reward is the 
discovery of the city mainly via sound. Success depends 
on his listening behavior: N.P.C. are moving around him, 
interpreting his moves, the direction he takes and the time 
spend listening particular sounds. Depending upon the 
listener’s attitude, each N.P.C. has his own reaction such 


as running away, getting closer to the listener, ignoring 
him or helping him to discover secret areas. 


3.1 Sound design choices 


The starting point of this project is the dynamic behavior 
of sound in space and the importance of the auditory 
attention that is arroused in the virtual city. In this first 
experiment, we classified the sounds according to the 
following roles: 

e Landscape sounds: These are ambient sounds 
that contribute to the interactor’s sense of 
immersion and the credibility of the scene in the 
case of some sounds. For example, the sounds of 
wind or traffic reposition the interactor in a 
familiar situation, physically experienced in a 
real city. Other sounds, more akin to urban 
drones, contribute to this immersion but in a way 
that is more abstract and perhaps less conscious. 

e Clue sounds: These are very localized, with the 
goal of attracting the attention of the interactor’s 
avatar and to induce him to move to a specific 
point in the city. These clue sounds disappear as 
soon as the avatar leaves a zone in which they 
are located. Clue Sounds are not tied to any 
fixed or animated visual entity. They exist only 
in the auditory space of the city map. 

e Object sounds: These are attached to a visual 
object, generally also to its behavior (eg, the 
appearance of a building, speaking of a 
character). The object sounds show that the 
interactor’s avatar has triggered an event such as 
the appearance of a building, a character or a 
sound - even while outside of the interactor’s 
field of view - and draws his attention towards 
them. 

At the beginning of the piece, the interactor starts 
almost in an empty black and white landscape.The city is 
revealed by way of outlines traced by its streets and 
buildings. The challenge to its quasi non-existence at the 
start of the interactor's experience is essential. The loss of 
visual cues encourages hearing acuity, and the gradual 
emergence of streets and buildings at once bolsters the 
walker in a known domain and spatially confines him in a 
way that is almost maze-like. This city, while at first 
invisible to the interactor, bears all of the spatial 
constraints that the interactor will meet thereafter. 
Although the streets and buildings are invisible, the 
locations of sounds, for example, are closely related to 
the city map. 


3.2 Development process aspects of the scripted 
version 


The scripted version was designed using a methodology 
strongly inspired by the level design of adventure games 
and the maze principle [14]. The interactor’s avatar 
appears at a given initial point of an empty map. Invisible 
barriers and the associated triggers are placed according 
to possible paths of the interactor’s avatar. The paths and 
the triggers are partially ordered to create sound events 
and the revelation of roads and buildings. Triggers are 


also used to guide the interactor: they lead to sound 
events located in the space. N.P.C., according to scripted 
behavior, try to seduce or intrigue the interactor. As the 
city is progressively revealed, the path of the avatar 
becomes more and more constraint by the walls of the 
building. The story ends when the last trigger is activated, 
revealing the Pantheon. 

The mapping between triggers, events and N.P.C.’s 
behavior is written using the graphical programming 
language of the CryEngine, the Flowgraph. 


Figure 3. City 4 map in the CryEngine editor 


Let us illustrate this method through the City 4 version 
of The Listening Walker. City 4 is a constrained itinerary 
in which the interactor, hearing a sound, starts from an 
empty space and gradually built the city. Invisible 
barriers are set up on the space, as interaction spots. They 
set off events when the interactor enters in their influence 
area. A trigger can set off a sound, create a new road or a 
new building. These triggers are also logically ordered. 
Some of them can only be activated if the interactor came 
across the previous ones. This logical scripting ensures 
that the events occur in a non-anarchic way and the 
consistency of the itinerary. 


las the player appears at the intersection of rue Saint-Jacques 
and rue Soufflot, 
4 sound hidden in rue Saint-Jacques 2 is activated; 
When the player arrives over there, 

A part of the urban outline of a district becomes visible; 
(see FlowGraph /entitic orA 1 ors T teon). 
The sound of rue Saint-Jacques 2 deactivated 
and a bass tone comes up rue d’Ulm; 
An NPC - Le Chevalier — appears visually, read a text and 
heads towards the back of the Pantheon following a given path; 
If the player follows him, he triggers a sound moving around him; 
If the player stays longer enough listening the moving sound, 
two more sound entities are triggered; 
If the player tries to catch them at the intersection 
of rue Clotaire and place du Pantheon, they vanish; 
INPC Bakal appears place du Pantheon, talking to himself. 


Figure 4. Example of City 4 map scripting 


For each sound rendering entity (e.g. Fond vent.ogg), we 
have listed its trigger conditions (e.g. triggered at start- 
up), its playback mode (e.g. once), the stop conditions 
(e.g. “when the avatar passes through the St. Jacques 
trigger”), name of the associated flowgraph (e.g. 
“SoundDisapear: ZoneVentVille4”), geographic location 
of activation (e.g. the entire map), activity and 
attenuation zone (e.g. 0-150 meters), its spatial behavior 
(e.g. none), the noise level (e.g. constant) and the name of 
the entity in the CryEngine (eg Ambient volume + area 
shape). 

Sound objects can either be linked either to a virtual 
actor, an area of the city, or a building behavior. They can 
also serve as a clue for the interactor’s progress in the 


city. 


The following is an example of a sound object’s area of 
influence (Fig. 5). Within, the sound is audible with a 
possible attenuation cone from the center of the sphere, 
relative to the distance between the interactor and the 
position of the source of the sound. 


Figure 5. Ambient sound 


Figure 6 shows the trajectory of a sound event triggered 
by the collision of the interactor with the trigger for this 
event. The sound is then activated in the space and 
follows a path between the red points positioned with the 
designer on the map. 


Figure 6. Ambient sound 


4. CONCLUSIONS AND FUTURE WORKS 


In this paper we set out, throught the analysis of 
interactive art works, the different choices to set 
interactivity within the D.LS. triangle. These give rise to 
more or less autonomy to interactor or to Non Player 
Characters. A design choice relates to two writing styles. 
The design and the development of the scripted writing 
style is illustrated by the first version of The Listening 
Walker installation. In this conclusion, we present first 
the design-in-progress of the aforementioned piece 
according to the emergent style. We then tie our 
observations to interviews we conducted with designers 
mainly about the role they want to assume in their art 
works. Finaly we present our steps in our study on the 
design process of interactive sound walk installations. 


4.1 Emergent version of The Listening Walker 


Even if we used the same story, the same universe and 
the same technology as in its scripted version, the design 
of the emergent version of The Listening Walker is based 
on a completely different approach to the scripted one. 
The main principle used is that any action is decided and 
executed by an agent of the system, a Non Player 
Character. Every N.P.C. therefore has his own 


perception, decision making and action system. His 
behavior relies only on those events that he has been able 
to observe and memorize. Each N.P.C. is the narrator of 
its own version of the story. The narration relies on a 
partition of the space into areas. An area is controlled by 
an invisible stationary N.P.C. Moving N.P.C.s are 
permitted to act within defined sectors of the map. These 
moving N.P.C.s have the same names and the same 
personalities as in the scripted version. Moreover, the 
empathycy relationship between the interactor and a 
moving N.P.C. relies on the same  ideas.The 
corresponding actions and rules, however, are non 
contextual. In the emergent writing style, the designer 
describes what N.P.C.s perceive and how they are 
perceived by other N.P.C.s and by the interactor. Rather 
than positioning triggers as an ordered sequence of steps, 
writing here involves the development of a decision- 
making system for agents. 

The emergent version of The Listening Walker leads us 
to the use of automata in the CryEngine. The exploration 
of this engine has helped us to identify interesting 
possibilities for our own practice through the methods 
used in video games. Methodological implications of our 
work are derive from the dramatization of the space, the 
writing of rules of interaction and behavior for 
autonomous agents that combinine procedural logic and 
object-oriented programming. 

The entire design process of the emergent version and 
the description of our comparison between the two style 
can be found in [1]. The outline of this conclusion: it is 
easier to write a story, even an interactive one, than to 
design a world. From an intellectual and even an artistic 
point of view, the possibilities opened up by the 
emergent style are very promising, though this way to 
conceptualize an artwork as a set of localized and 
dynamic pieces is a break with the common practice of 
music composers. The scripted style is still a much more 
comfortable way to design and develop an interactive 
sound walk. 


4.2 Supplementary survey and interviews 


Our analysis takes the point of view of the designer, and 
more specifically one of the authors of this paper. As a 
consequence, they cannot be directly generalized. We 
prepared an interviews for seven well-known designers 
coming from various fields (Samuel Bianchini, Xavier 
Boissarie, Roland Cahen, Luc Courchesne, Jean-Marie 
Dallet, Mikhail Malt, Eric Viennot). The complete result 
of this study can be found in [1] and seems to confirm 
our conclusions. Here is a summary of our main 
reflections. 

We have established a global assessment by linking our 
own observations, outlined in the previous paragraph, to 
interviews conducted with developers of interactive 
works. Our investigation takes as its starting point the 
role that the designer claims to perform. The survey then 
tries to clarify the different interests and approaches of 
the notion of interactive pieces according to each artist’s 
point of view. 


Our survey reveals some trends of thought common to 
almost all artists. The interest in the design of interactive 
art works is to commit the interactor, to establish a co- 
dessign relationship and a dialogue between the interactor 
and the art work. This results to bring to the interactor 
something of an aesthetic experience that can not exist 
without this relationship. 

As an example, consider the following sentences of Eric 
Viennot, the game designer of In Memoriam, one of the 
first Alternate Reality Game: 

“At a given time, as a designer, we are creating a 
universe where some emerging part of the story may 
appear. It is something that we did not really foresee. It is 
really interesting and I call it emergent narration... 

I like to master the time when the player feels a given 
emotion. I think that we must find in some way a mixed 
solution: too open works miss some artistic dimension.” 

From the point of view of designers of interactive art 
works, there is no antinomy between control and 
autonomy. This conclusion relies on several reasons: the 
designer remains the designer unless he resigns, the 
feeling of the interactor’s engagement can be created 
even in strongly driven interactive pieces, autonomous 
systems produce interesting situations if the control 
parameters are precisely choosen and intelligible to the 
interactor. They also all agreed on the fact that the 
emergent style seems to be more interesting but certainly 
much more difficult than the scripted one. 

In summary, adopting the point of view of the interactor 
or the system at the time of design changes the process, 
but does not change the role of the designer. 


4.3 Future Works 


The comparison of the two versions of the same piece 
introduced here takes only the point of view of the 
composer. Until know, only the scripted version has been 
exhibited. As the emergent version is currently in 
development, it will be possible, by conducting 
interviews with the public, to have also the point of view 
of the interactor. Of course the scope of the results is 
limited to this piece but it is probably a unique 
experiment, as the same piece is rarely developed twice. 
From our personal point of view, we assume the risk to 
feel insecure as a composer moving our way of thinking 
from time to space and from determinism to emergence 
and interactivity. 

“It is when artists combine the computer’s capabilities 
of real-time autonomy and reactivity that they achieve a 
deeper form of interactive art. By making the computer 
listen to the audience (the first half of reactivity), think 
about what it heard (autonomy), and then speak its 
thoughts back to the audience (the second half of 
reactivity), the artwork can have a dialog, a 
conversation, with the audience. (By “speaking” and 
“conversation” I mean some sort of meaningful 
communication, not necessarily literal speech.) 
Interactivity is the cycle where both the artwork and the 
audience listen, think and speak to each other. “ 

Andrew Stern [15] 
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